1. Technical Field
The present invention relates to evaluation of program, and more particularly to systems and methods which determine program soundness.
2. Description of the Related Art
The soundness of analyses and algorithms has been a cornerstone in programming languages. A static analysis is said to be sound when the data flow information that it produces is guaranteed to be true on all program execution. Foundational theories like Meet-Over-All-Paths (MOAP) and Abstract Interpretation are the basis for sound static analysis.
Unfortunately, scalability becomes an issue when implementing precise and sound static analysis. Soundness of static analysis is extremely important for many applications, including code generation and code transformation. On the other hand, there are applications of static analysis where soundness is not very critical, such as for tracking bugs and defects. One can potentially construct a very simple and fast unsound static analysis to compute data flow information. Unfortunately, unsound static analysis can produce misleading data flow information, and in the context of bug finding tools, can potentially generate many false negatives and false positives.
Traditional static analysis based on foundational theories like the Meet-Over-All-Paths (MOAP), Maximal Fixed Point (MFP), and Abstract Interpretation may guarantee soundness of the static analysis. A static analysis is said to be sound when the data flow information that it produces is guaranteed to be true on all program execution. Soundness of static analysis is extremely important and critical for many applications of static analysis, such as code generation and optimization. There are applications of static analysis for which one can sacrifice soundness. For example, one can do away with soundness for finding bugs and defects.
A negative consequence of loosing soundness in such applications is increased reporting of false negatives. In software diagnosis, as in medical diagnosis, there are two kinds of error that can occur. A false positive is when there is no bug, but the results of the diagnosis come back as positive. A false negative is when there is actually a bug, but the results of diagnosis come back as negative. If a static analysis is unsound, the number of false negatives and sometimes even the number of false positives can be increased.
One major stumbling block in (sound) static analysis is the tradeoff between scalability and precision of the analysis. Typically, a scalable analysis is often less precise. Precision of the analysis has direct impact on the number of false positives. Precise static analysis often reduces the number of false positives.
Doing away with the soundness property of static analysis has been proposed to achieve scalability. In one such method, random interpretation is used for solving certain data flow problems and the resulting solution is not necessarily sound with respect deterministic MOAP. Another technique was presented in the context of model checking in SLAM for localizing faults in error traces by exploiting existence of correct traces. Delta debugging uses a correct program and a series of changes to the program to isolate the fault that caused the error.
A static analysis tool called PSE was employed to diagnose software failures. Given minimal information about a particular program failure, PSE automatically reconstructed a set of failing execution traces. PSE requires the specification of the failure in the form of type state information. An unsound pointer analysis was proposed which assumed that pointers passed into a procedure, in parameters, in global variables, and locations reachable from variables are all distinct.
Unsound static analysis techniques have been used on a number of bug patterns in programs for detecting such patterns. Unsound static analysis techniques have also been employed for finding a class of error-handling mistakes that arise from an improper release of resources.
Program slicing is another technique that is useful for detecting bugs and defects. A program slice is a set of all program statements that affect the value of a variable. In dynamic slicing only those statements that affect a particular test case are considered. Slicing focuses on finding statements that affect a particular value of a variable.